modularize the config module bootstrap #141272

Shourya742 · 2025-05-20T03:14:31Z

Currently, our config module is quite large over 3,000 lines, and handles a wide range of responsibilities. This PR aims to break it down into smaller, more focused submodules to improve readability and maintainability:

toml: Introduces a dedicated toml submodule within the config module. Its sole purpose is to define configuration-related structs along with their corresponding deserialization logic. It also contains the parse_inner method, which serves as the central function for extracting relevant information from the TOML structs and constructing the final configuration.
rust, dist, install, llvm, build, gcc, and others: Each of these modules contains TOML subsections specific to their domain, along with the logic necessary to convert them into parts of the final configuration struct.
config/mod.rs: Contains shared types and enums used across multiple TOML subsections.
config/config.rs: Houses the logic that integrates all the TOML subsections into the complete configuration struct.

r? @Kobzol

rustbot · 2025-05-20T03:14:37Z

This PR modifies src/bootstrap/src/core/config.

If appropriate, please update CONFIG_CHANGE_HISTORY in src/bootstrap/src/utils/change_tracker.rs.

Kobzol

I like the general direction of this. Could you perhaps split the changes into multiple commits though? It would be easier to review. Thank you :)

src/bootstrap/src/core/config/toml.rs

Kobzol · 2025-05-22T08:00:30Z

Kind of crazy just how much code is there in config definition and parsing. The changes LGTM, but since this is a large-ish change, I'd like to hear also from other members of t-bootstrap if they are OK with it. CC @onur-ozkan @jieyouxu.

src/bootstrap/src/core/config/toml.rs

src/bootstrap/src/core/config/types.rs

onur-ozkan · 2025-05-24T10:25:10Z

@rustbot author

rustbot · 2025-05-24T10:25:16Z

Reminder, once the PR becomes ready for a review, use @rustbot ready.

jieyouxu · 2025-05-24T10:29:15Z

cc @clubby789

Shourya742 · 2025-05-24T15:23:25Z

@rustbot review

jieyouxu · 2025-05-26T11:55:28Z

Vibes-wise, organization seems fine to me.

onur-ozkan · 2025-05-28T07:55:09Z

I still have this concern (also see this as an additional ref). With the current approach, adding new things will always be unclear for some people. Even now, for example, we have parsing.rs and types.rs modules but some parsing logic has leaked into other modules and some types have leaked to some other modules as well.

What I would suggest instead is organizing the modules based on the configuration field. For example, mod.rs could represent the root level of the configuration and the rest could be split into the sub fields like rust.rs for [rust], llvm.rs for [llvm], target_specific.rs for things like [target.x86_64-unknown-linux-gnu], etc.

With this structure it would make it very explicit where things belong and anything that doesn't fit this rule can be easily caught during PR reviews.

src/bootstrap/src/core/config/toml/macros.rs

onur-ozkan

LGTM. Thank you so much for looking into this!

onur-ozkan · 2025-06-04T04:23:19Z

Do you have any concerns on the current state of this PR? I don't want to r+ without your approval since you were the main reviewer. @Kobzol

Kobzol · 2025-06-04T06:20:39Z

No, I'm good. Thank you, great work!

@bors r+

bors · 2025-06-04T06:20:42Z

📌 Commit 6ec7ec9 has been approved by Kobzol

It is now in the queue for this repository.

…onfig-module, r=Kobzol modularize the config module bootstrap Currently, our `config` module is quite large over 3,000 lines, and handles a wide range of responsibilities. This PR aims to break it down into smaller, more focused submodules to improve readability and maintainability: * **`toml`**: Introduces a dedicated `toml` submodule within the `config` module. Its sole purpose is to define configuration-related structs along with their corresponding deserialization logic. It also contains the `parse_inner` method, which serves as the central function for extracting relevant information from the TOML structs and constructing the final configuration. * **`rust`, `dist`, `install`, `llvm`, `build`, `gcc`, and others**: Each of these modules contains TOML subsections specific to their domain, along with the logic necessary to convert them into parts of the final configuration struct. * **`config/mod.rs`**: Contains shared types and enums used across multiple TOML subsections. * **`config/config.rs`**: Houses the logic that integrates all the TOML subsections into the complete configuration struct. r? `@kobzol`

Rollup of 8 pull requests Successful merges: - #140638 (UnsafePinned: also include the effects of UnsafeCell) - #141272 (modularize the config module bootstrap) - #141777 (Do not run PGO/BOLT in x64 Linux alt builds) - #141870 (Fix broken link to rustc_type_ir module in serialization docs) - #141938 (update rust offload bootstrap) - #141962 (rustc-dev-guide subtree update) - #141965 (`tests/ui`: A New Order [3/N]) - #141970 (implement new `x` flag: `--skip-std-check-if-no-download-rustc`) r? `@ghost` `@rustbot` modify labels: rollup

matthiaskrgr · 2025-06-05T10:20:49Z

@bors r-
#142059 (comment)

Kobzol · 2025-06-05T11:12:16Z

Hmm, we should probably run x doc bootstrap in PR CI, this is not the first time this has bitten us, and we will hopefully document more and more things in bootstrap now.

@bors r+ rollup=iffy

bors · 2025-06-05T11:12:18Z

📌 Commit 3667dbd has been approved by Kobzol

It is now in the queue for this repository.

bors · 2025-06-06T04:32:59Z

⌛ Testing commit 3667dbd with merge d00435f...

bors · 2025-06-06T07:33:31Z

☀️ Test successful - checks-actions
Approved by: Kobzol
Pushing d00435f to master...

github-actions · 2025-06-06T07:36:39Z

What is this?

This is an experimental post-merge analysis report that shows differences in test outcomes between the merged PR and its parent PR.

Comparing cf42371 (parent) -> d00435f (this PR)

Test differences

No test diffs found

Test dashboard

Run

cargo run --manifest-path src/ci/citool/Cargo.toml -- \
    test-dashboard d00435f223dc3a88d8c5f472b10ba948b7959cc6 --output-dir test-dashboard

And then open test-dashboard/index.html in your browser to see an overview of all executed tests.

Job duration changes

dist-apple-various: 6377.8s -> 8058.3s (26.4%)
aarch64-apple: 4692.3s -> 4026.0s (-14.2%)
x86_64-gnu: 6636.4s -> 7492.5s (12.9%)
x86_64-rust-for-linux: 2548.0s -> 2859.1s (12.2%)
armhf-gnu: 4482.8s -> 5028.1s (12.2%)
i686-gnu-1: 7149.2s -> 8018.6s (12.2%)
i686-gnu-nopt-1: 7074.8s -> 7919.2s (11.9%)
i686-gnu-2: 5285.6s -> 5902.9s (11.7%)
x86_64-gnu-llvm-20-1: 3226.9s -> 3587.3s (11.2%)
mingw-check-2: 1836.4s -> 2040.4s (11.1%)

How to interpret the job duration changes?

Job durations can vary a lot, based on the actual runner instance
that executed the job, system noise, invalidated caches, etc. The table above is provided
mostly for t-infra members, for simpler debugging of potential CI slow-downs.

rustbot assigned Kobzol May 20, 2025

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) labels May 20, 2025

This comment has been minimized.

Sign in to view

Shourya742 force-pushed the 2025-05-18-modularize-config-module branch from 6a1e868 to b8374d6 Compare May 20, 2025 03:17

This comment has been minimized.

Sign in to view

Kobzol reviewed May 20, 2025

View reviewed changes

src/bootstrap/src/core/config/toml.rs Outdated Show resolved Hide resolved

Shourya742 force-pushed the 2025-05-18-modularize-config-module branch from b8374d6 to ad21211 Compare May 21, 2025 17:19

jieyouxu self-assigned this May 24, 2025

onur-ozkan reviewed May 24, 2025

View reviewed changes

src/bootstrap/src/core/config/toml.rs Outdated Show resolved Hide resolved

onur-ozkan reviewed May 24, 2025

View reviewed changes

src/bootstrap/src/core/config/types.rs Outdated Show resolved Hide resolved

onur-ozkan reviewed May 24, 2025

View reviewed changes

src/bootstrap/src/core/config/types.rs Outdated Show resolved Hide resolved

rustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels May 24, 2025

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels May 24, 2025

jieyouxu removed their assignment May 26, 2025

Shourya742 requested review from Kobzol and onur-ozkan May 29, 2025 12:42

This comment has been minimized.

Sign in to view

Shourya742 force-pushed the 2025-05-18-modularize-config-module branch from 2dc25e6 to 877aa18 Compare May 29, 2025 12:53

onur-ozkan reviewed Jun 1, 2025

View reviewed changes

src/bootstrap/src/core/config/toml/macros.rs Outdated Show resolved Hide resolved

Shourya742 added 3 commits June 3, 2025 13:46

move target selection to its separate module

cc30123

Move all common types, macros and utility method to config/mod.rs

0de60dd

add config.rs with new module structuring

edada36

Shourya742 force-pushed the 2025-05-18-modularize-config-module branch from 1ec9ad3 to 6ec7ec9 Compare June 3, 2025 08:26

onur-ozkan approved these changes Jun 4, 2025

View reviewed changes

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jun 4, 2025

matthiaskrgr mentioned this pull request Jun 5, 2025

Rollup of 8 pull requests #142059

Closed

bors added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Jun 5, 2025

correct the imports in flags, tests and download

3667dbd

Shourya742 force-pushed the 2025-05-18-modularize-config-module branch from 6ec7ec9 to 3667dbd Compare June 5, 2025 10:57

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Jun 5, 2025

Kobzol mentioned this pull request Jun 5, 2025

Check documentation of bootstrap in PR CI #142076

Open

bors added the merged-by-bors This PR was explicitly merged by bors. label Jun 6, 2025

bors merged commit d00435f into rust-lang:master Jun 6, 2025
11 checks passed

rustbot added this to the 1.89.0 milestone Jun 6, 2025

bors mentioned this pull request Jun 6, 2025

Use lld by default on x86_64-unknown-linux-gnu stable #140525

Open

modularize the config module bootstrap #141272

modularize the config module bootstrap #141272

Conversation

Shourya742 commented May 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rustbot commented May 20, 2025

Uh oh!

This comment has been minimized.

This comment has been minimized.

Kobzol left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Kobzol commented May 22, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

onur-ozkan commented May 24, 2025

Uh oh!

rustbot commented May 24, 2025

Uh oh!

jieyouxu commented May 24, 2025

Uh oh!

Shourya742 commented May 24, 2025

Uh oh!

jieyouxu commented May 26, 2025

Uh oh!

onur-ozkan commented May 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment has been minimized.

Uh oh!

onur-ozkan left a comment

Choose a reason for hiding this comment

Uh oh!

onur-ozkan commented Jun 4, 2025

Uh oh!

Kobzol commented Jun 4, 2025

Uh oh!

bors commented Jun 4, 2025

Uh oh!

matthiaskrgr commented Jun 5, 2025

Uh oh!

Kobzol commented Jun 5, 2025

Uh oh!

bors commented Jun 5, 2025

Uh oh!

bors commented Jun 6, 2025

Uh oh!

bors commented Jun 6, 2025

Uh oh!

Uh oh!

github-actions bot commented Jun 6, 2025

Test differences

Job duration changes

Uh oh!

Uh oh!

Shourya742 commented May 20, 2025 •

edited

Loading

onur-ozkan commented May 28, 2025 •

edited

Loading